<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
    <meta name="resource-type" content="document">
    <meta name="description" content="The OpenBSD Port Testing Guide">
    <meta name="keywords" content="openbsd,ports, test">
    <meta name="distribution" content="global">
    <meta name="copyright" content="copyright 2002-2004 by OpenBSD.">
    <title>OpenBSD Port Testing Guide</title>
  </head>

  <body text="#000000" bgcolor="#FFFFFF" link="#23238E">
    <a href="index.html"><img alt="[OpenBSD]" height="30" width="141" src=
    "images/smalltitle.gif" border="0"></a> 

    <h2><font color="#e00000">Port Testing Guide</font></h2>
    <hr>

    <h2><font color="#0000e0">Index</font></h2>

    <ul>
      <li><a href="#Introduction">Introduction</a></li>

      <li><a href="#How">How</a></li>

      <li><a href="#First">First step</a></li>

      <li><a href="#Testing">Testing</a></li>

      <li><a href="#Commenting">Commenting</a></li>

      <li><a href="#More">More testing</a></li>
    </ul>
    <hr>

    <h2><a name="Introduction"><font color=
    "#0000e0">Introduction</font></a></h2>

    <p>The <a href="faq/faq15.html#Ports">ports tree</a> is a huge piece
    of work that
    permits OpenBSD users to use third-party programs without wasting time 
    patching, configuring and installing each one individually. This work 
    is done by a group of volunteers who spend their time porting and testing
    applications across the range of OpenBSD platforms. Many people think 
    that they cannot help this process because they do not have enough 
    knowledge; this is false because they can help porters work better and 
    faster.</p>

    <h2><a name="How"><font color="#0000e0">How</font></a></h2>

    <p>Testing submitted updates or new ports which are posted on the
    <a href="mail.html">ports mailing list</a>. By doing this
    you reduce the latency of commits and also increase the number of ports to
    be committed (many ports are not committed because of lack of testing!).</p>

    <h2><a name="First"><font color="#0000e0">First step</font></a></h2>

    <p>The ports tree is developed against 
    <a href="plus.html">OpenBSD-current</a>; there is no guarantee that 
    new ports or updates will work correctly on the other branches.
    This means 
    you should upgrade your system and ports tree to -current 
    (instructions on how to do this can be found at the 
    <a href="anoncvs.html">anoncvs</a> and 
    <a href="faq/current.html">Following Current</a>) pages.
    It is also recommended that you subscribe to the ports and ports-changes
    <a href="mail.html">mailing-lists</a>. This way you will be notified
    about new or updated ports and about changes in the ports tree.</p>
    
    <h2><a name="Testing"><font color="#0000e0">Testing</font></a></h2>

    <p>There are two types of submissions on the mailing-lists; new ports
    and updates. New ports are generally posted as tarball attachments
    or URLs. A good idea is to extract them into the 
    <tt>/usr/ports/mystuff/</tt> directory and test from there.
    Updates are generally a diff against the -current ports tree, so it is best 
    to copy the port to <tt>mystuff/</tt> and apply the diff to prevent tree
    breakage.</p>

    <p>Step-by-step building is needed to verify that every target, see 
    <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=ports&amp;sektion=7&amp;">
    ports(7)</a>, is achieved correctly :</p>

    <ul>
      <li><i>fetch</i>
        <ul>
         <li>Needed to verify that distfile(s) are correctly downloaded. Try
         to test all of the <tt>MASTER_SITES</tt> specified to make sure they
         are all valid sources.</li>
        </ul>

      <li><i>checksum</i>
        <ul>
         <li>Verify that downloaded distfiles match the checksums recorded
         in the <tt>distinfo</tt> file.</li>
        </ul>

      <li><i>extract</i>
        <ul><li>Each of the distfiles should install any dependent programs
        (such as bzip2) beforehand.</li></ul>

      <li><i>patch</i>
        <ul>
         <li>Patches should apply cleanly without any warnings.
         <li>There shouldn't be any <tt>.orig</tt> files left behind in the
             <tt>patches/</tt> directory.
         <li>Another common mistake is to include RCS tags in a patch; this
         will break when the port is checked into the repository and the RCS
         tag expanded.
        </ul>

      <li><i>configure</i>
        <ul>
         <li>Check that configure scripts correctly detect features
             on your platform.
         <li>GNU libtool is notorious for undesired 'features' on OpenBSD,
             so check that the port is using a recent version which creates
             correct libraries on your architecture.
         <li>The configure script should not detect stray applications already
             installed on your system without explicit dependencies being
             set in the port.
        </ul>

      <li><i>build</i>
        <ul>
         <li>Check for build errors and suspicious warnings.
         <li>Warnings about <a href=
      "http://www.openbsd.org/cgi-bin/man.cgi?query=tmpnam&amp;sektion=3&amp;">
      tmpnam(3)</a> issues should be resolved by using
      <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=mkstemp&amp;sektion=3&amp;">
          mkstemp(3)</a>.</li>
         <li>Try to set the <tt>SEPARATE_BUILD</tt> variable to 'concurrent'
         and test if the build still works.
         <li>Make sure dependencies on GNU make are really necessary.
         <li>Check whether the port is using libtool. If it is, try
         setting <tt>USE_LIBTOOL=Yes</tt>. This way, the ports
         infrastructure will use the libtool version in the tree, instead
         of libtool versions that come bundled with the software's source.
        </ul>

      <li><i>regress</i>
        <ul>
          <li>Check for errors (empty tests means okay).
        </ul>


      <li><i>fake</i>
        <ul>
         <li>This target installs the application into a fake working
         directory, to ensure that all files can be easily packaged up
         without affecting the base system.
         <li>The port should <b>never</b> install files outside of the
         fake directory (such as into <tt>/usr/local</tt>).
         This kind of mistake can be easily caught by using
         <tt>USE_SYSTRACE=Yes</tt>.
         <li>GNU libtool occasionally has trouble relinking libraries
         during the fake process on some architectures.
         <li>Check whether all files get installed with correct ownerships
         and permissions.
        </ul>

      <li><i>package</i>
        <ul>
         <li>Package creation can break if pkg/PLIST* and/or
         pkg/PFRAG* are wrong.</li>
        </ul>

      <li><i>lib-depends-check</i>
        <ul>
         <li>This will check whether all libraries on which the port depends
         can be reached through either <tt>LIB_DEPENDS</tt> or <tt>WANTLIB</tt>.
         The result should be empty. The above variables should be inspected
         when you see lines starting "Extra" or "Missing".
        </ul>

      <li><i>install</i>
        <ul>
         <li>Packages should install all of the files from their packaging
         lists successfully and with the correct permissions.  Be especially
         careful of files with the setuid bits set.
         <li>Make sure that the package <tt>INSTALL</tt> script works
         correctly, and does not overwrite any files in <tt>/etc</tt>.
        </ul>

      <li><i>deinstall</i>
        <ul>
         <li>This should remove all files installed by the package, except
         those in <tt>/etc</tt>.
         <li>Make sure that the <tt>DEINSTALL</tt> script works as expected.
        </ul>
    </ul>

    <p>Remaining <tt>pkg/</tt> files like DESCR and MESSAGE should be checked
    for grammar and typos.  Paragraphs should be formatted using
    <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=fmt&amp;sektion=1&amp;format=html">fmt(1)</a> and wrapped at 70 characters.</p>

    <h2><a name="Commenting"><font color="#0000e0">Commenting</font></a></h2>

    <p>At the end of the test comes the really important thing : comments.
    Even if the port is working fine comments must be done. If we have ten
    posts where people say that the port runs fine under different architectures
    then the commit is done faster. If it does not work then some information
    must be given.
    There are tools that can help in this task, like <a href=
    "http://www.openbsd.org/cgi-bin/cvsweb/ports/infrastructure/build/portslogger">
    portslogger</a> which is like an "intelligent tee" that redirects output
    into a log file.</p>

    <p>Example :<br>
    </p>

    <div style="margin-left: 2em">
      <table border="0" cellpadding="3" width="90%">
        <tr>
          <td nowrap bgcolor="#EEEEEE">
<pre>
    # make install 2&gt;&amp;1 | /usr/ports/infrastructure/build/portslogger .
</pre>
          </td>
        </tr>
      </table>
    </div>
    <br>
     <br>
     

    <p>This will redirect the output into a log file located in the current
    directory.</p>

    <p>Finally, once the port is found to be okay, other ports depending on
    it should also be tested, to check whether they are still working
    correctly. The <i>show-required-by</i> make target will help to find
    other ports which depend on the current one.

    <h2><a name="More"><font color="#0000e0">More testing</font></a></h2>

    <p>Check the port's Makefile for correct dependencies, typos, incorrect
    links, useless or missing variables, correct licensing and categories.
    Those who are more skilled can help by examining patches, as well as
    providing diffs to correct bugs, add flavors, or other enhancements.</p>
    <p>These diffs should be done with the <tt>-uNr --exclude=CVS</tt> options.
    <tt>cvs diff -uN</tt> can also be used to generate patches against
    the CVS repository.</p>
<hr>
<a href="index.html"><img src="back.gif" height="24" width="24" alt="OpenBSD" border="0"></a>
<a href="mailto:www@openbsd.org">www@openbsd.org</a><br>
<small>
$OpenBSD: porttest.html,v 1.20 2005/12/20 08:33:09 steven Exp $
</small>
</body>
</html>
